Multimodal biometric authentication system and method with photoplethysmography (PPG) bulk absorption biometric

ABSTRACT

A multimodal biometric authentication system utilizes a bulk absorption characteristic of human tissue that is measurable using a photoplethysmography (PPG) sensor. One disclosed method of operation includes extracting bulk absorption features from biometric data obtained using a PPG sensor and generating a first biometric template. Additional biometric features are also extracted from biometric data obtained using a second biometric sensor and a second biometric template is also generated. An authentication output signal is provided in response to the first biometric template matching a first stored corresponding enrolled biometric template and the second biometric template matching a second stored corresponding enrolled biometric template.

FIELD OF THE DISCLOSURE

The present disclosure relates generally to biometric authenticationsystems and more particularly to biometric fusion (also referred to as“multimodal”) authentication systems that use two or more biometrics orthat use two or more algorithms applied to the same biometric data.

BACKGROUND

Biometric authentication systems offer a quick and convenient means ofpersonal authentication without the need to remember and enterpasswords. Various forms of biometric authentication have been developedand each form presents its own unique set of challenges. Generallyspeaking, biometric authentication systems are subject to spoofing orfalsing.

For example, fingerprint biometric authentication systems can sometimesbe subject to falsing by an unauthorized person's finger or may even be“fooled” by a synthetic false finger. This susceptibility to falsing isan issue with many types of biometric authentication or identificationsystems. Because of this susceptibility, the concept of “biometricfusion” or “multimodal biometric” systems is gaining in popularity. Insuch multimodal systems, it becomes more difficult to spoof or fake thebiometric inputs because there is either more than one biometricrequired, or multiple algorithms are applied to the same biometric, andtherefore the probability of spoofing to gain unauthorized access issignificantly reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a photoplethysmography authenticationsystem in accordance with an embodiment.

FIG. 2 is a flow chart showing a method of operation of the system shownin FIG. 1, in accordance with an embodiment.

FIG. 3 is an example plot of photoplethysmography sensor voltage versusLED current which corresponds to bulk absorption for various users.

FIG. 4 is a flowchart showing a method of operation using either amultispectral photoplethysmography sensor or using a first sensor havinga first spectral range and a second sensor having a second spectralrange in accordance with an embodiment.

FIG. 5 is a block diagram of a multimodal biometric authenticationsystem with a first device having two biometric sensors and a seconddevice performing authentication data processing in accordance with anembodiment.

FIG. 6 is a flow chart showing a method of operation of the system shownin FIG. 5, in accordance with an embodiment.

FIG. 7 is a block diagram of a multimodal biometric authenticationsystem with a first device having two biometric sensors and a performingauthentication data processing by accessing a database in a seconddevice in accordance with an embodiment.

FIG. 8 is a flow chart showing a method of operation of the system shownin FIG. 7, in accordance with an embodiment.

FIG. 9 is a block diagram of a multimodal biometric authenticationsystem with a first device having a first biometric sensor,authentication data processing, and a first database. A second devicehas a second biometric sensor, authentication data processing and asecond database. The first device and the second device exchangeauthentication information in accordance with an embodiment.

FIG. 10 is a flow chart showing a method of operation of the systemshown in FIG. 9, in accordance with an embodiment.

FIG. 11 is a block diagram of a multimodal biometric authenticationsystem in a device having a photoplethysmography biometric sensor, atleast one microphone operatively coupled to a voice recognition engineand authentication data processing in accordance with an embodiment.

FIG. 12 is a flow chart showing a method of operation of the systemshown in FIG. 11, in accordance with an embodiment.

DETAILED DESCRIPTION

The present disclosure provides multimodal biometric authenticationsystems utilizing a bulk absorption characteristic of human tissue thatis measurable using a photoplethysmography (PPG) sensor. The bulkabsorption characteristic can be measured using reflective PPGtechniques and sensors incorporated into wearable devices, mobile deviceor combinations thereof.

An aspect of the present disclosure is a method that includes extractingbulk absorption features from biometric data obtained using a PPG sensorand generating a first biometric template. Additional biometric featuresare also extracted from biometric data obtained using a second biometricsensor and a second biometric template is also generated. Accordingly,an authentication output signal is provided in response to the firstbiometric template matching a first stored corresponding enrolledbiometric template and the second biometric template matching a secondstored corresponding enrolled biometric template. In some embodiments,the second biometric sensor may be a second PPG sensor that operates ina different spectral range from the first PPG sensor. In suchembodiments, a multimodal biometric authentication system may beimplemented by requiring the bulk absorption characteristics, measuredwithin different spectral ranges, to match in order to provide anauthentication output signal. In yet other embodiments, the combinedbulk absorption characteristics at two or more spectral ranges may berequired, along with another different biometric.

In another aspect of the present disclosure, a method includesextracting bulk absorption features from biometric data obtained using aPPG sensor in a first device and generating a first biometric templatein the first device; sending a first authentication output signal, fromthe first device to a second device over a wireless link, in response tothe first biometric template matching a first stored correspondingenrolled biometric template stored in the first device; extractingbiometric features from biometric data obtained using a second biometricsensor in a second device and generating a second biometric template inthe second device; and providing a second authentication output signalin response to receiving the first authentication output signal from thefirst device, and the second biometric template matching a second storedcorresponding enrolled biometric template stored in the second device.

Another aspect of the present disclosure is a multimodal biometricauthentication apparatus that includes a PPG sensor; a second biometricsensor; non-volatile, non-transitory memory, containing at least twoenrolled biometric templates, a first enrolled biometric templaterelated to a bulk absorption biometric and a second enrolled biometrictemplate related a biometric measurable by the second biometric sensor;and a processor, operatively coupled to the PPG sensor, to the secondbiometric sensor, and to the memory.

The processor is operative to extract bulk absorption features frombiometric data obtained using the PPG sensor and generate a firstbiometric template; extract biometric features from biometric dataobtained using the second biometric sensor and generate a secondbiometric template; and provide an authentication output signal inresponse to the first biometric template matching the first enrolledbiometric template and the second biometric template matching the secondenrolled biometric template. In some embodiments, the second biometricsensor may be a second PPG sensor having a spectral range different thanthe first PPG sensor.

Another aspect of the present disclosure is a wearable device thatincludes a PPG sensor; a second biometric sensor; a transceiver; and acontroller, operatively coupled to the PPG sensor, to the secondbiometric sensor, and to the transceiver, The controller is operative toobtain biometric data collected using the PPG sensor and using thesecond biometric sensor; and send the biometric data obtained using thePPG sensor and using the second biometric sensor to a second device overa wireless link using the transceiver.

The wearable device may be included in a multimodal biometricauthentication system along with a mobile device. The mobile deviceincludes a transceiver; a controller, operatively coupled to thetransceiver; non-volatile, non-transitory memory, containing at leasttwo enrolled biometric templates, with a first enrolled biometrictemplate related to a bulk absorption biometric and with a secondenrolled biometric template related to a biometric measurable by thesecond biometric sensor in the wearable device; and data processinglogic, operatively coupled to the controller, The data processing logicis operative to extract bulk absorption features from biometric dataobtained using the PPG sensor in the wearable device and generate afirst biometric template; extract biometric features from biometric dataobtained using the second biometric sensor in the wearable device andgenerate a second biometric template; and provide an authenticationoutput signal in response to the first biometric template matching thefirst enrolled biometric template and the second biometric templatematching the second enrolled biometric template.

Another aspect of the present disclosure is a wearable device thatincludes a PPG sensor; a second biometric sensor; a transceiver; anddata processing logic, operatively coupled to the PPG sensor and to thesecond biometric sensor. The data processing logic is operative toextract bulk absorption features from biometric data obtained using thePPG sensor and generate a first biometric template; and extractbiometric features from biometric data obtained using the secondbiometric sensor and generate a second biometric template. A controlleris operatively coupled to the data processing logic, and to thetransceiver. The controller is operative to send the first biometrictemplate and the second biometric template to a second device over awireless link using the transceiver.

The wearable device may be included in a multimodal biometricauthentication along with a mobile device. The mobile device includes atransceiver; non-volatile, non-transitory memory, containing at leasttwo enrolled biometric templates: a first enrolled biometric templaterelated to a bulk absorption biometric and a second enrolled biometrictemplate related to a biometric measurable by the second biometricsensor in the wearable device. A controller is operatively coupled tothe transceiver and to the memory. The controller is operative toprovide an authentication output signal in response to the firstbiometric template matching the first enrolled biometric template andthe second biometric template matching the second enrolled biometrictemplate. The controller is further operative to send the authenticationoutput signal to either the wearable device or to an external thirddevice over a wireless link using the transceiver in the mobile device.

Another aspect of the present disclosure is a multimodal biometricauthentication system that includes a first device and a second device.The first device and the second device each include a biometric sensor;a transceiver; non-volatile, non-transitory memory that contains atleast one enrolled biometric template, with at least one of the first orthe second devices having a first enrolled biometric template related toa bulk absorption biometric and the other of the first or second deviceshaving a second enrolled biometric template; and data processing logic,operatively coupled to the biometric sensor.

The data processing logic is operative to extract bulk absorptionfeatures from biometric data obtained using the biometric sensor andgenerate a first biometric template. A controller is operatively coupledto the data processing logic, and to the transceiver. The controller isoperative to provide an authentication output signal in response to thebiometric template matching the at least one enrolled biometrictemplate; and send the authentication output signal to the other deviceover a wireless link using the transceiver.

In embodiments of the multimodal biometric authentication system, thebiometric sensor of the first device is a PPG sensor; and the biometricsensor of the second device may be either a microphone, a fingerprintsensor, a palm reader, or an iris scanner. In other embodiments, thebiometric sensor of the first device is a PPG sensor having a firstspectral range and the biometric sensor of the second device is a PPGsensor having a second spectral range.

Turning now to the drawings wherein like numerals represent likecomponents, FIG. 1 is a block diagram of a multimodal biometricauthentication system 100 that includes a photoplethysmography (PPG)bulk absorption biometric in accordance with an embodiment. Theauthentication system 100 includes a PPG sensor 101 and a secondbiometric sensor 103 which may be any of various types of biometricsensors. For example, the second biometric sensor 103 may be afingerprint reader, a palm print reader, an iris scanner, one or moremicrophones operatively coupled to a voice recognition system, or asecond PPG sensor in some embodiments. The PPG sensor 101 and the secondbiometric sensor 103 are operatively coupled to a processor 105. Thevarious components and devices that are described herein as being“operatively coupled” refers to such operative coupling as that having,in some embodiments, one or more intermediate or intervening componentsthat may exist between, or along the connection path between two suchcomponents such that the components are understood to be operativelycoupled in that data or commands or control signals can be sent from oneto the other and vice versa.

The processor 105 obtains raw sensor data from each biometric sensor andextracts the relevant biometric features. The processor 105 is operativeto use the extracted biometric features to create correspondingbiometric templates for comparison with enrolled biometric templatesstored in an enrolled templates database 107. The database 107 containsone or more enrolled biometric templates related to one or more users ofthe multimodal biometric authentication system 100. An enrolledbiometric template is a biometric template that a user has voluntarilyprovided to the authentication system, using a separate biometrictemplate enrollment security procedure, to verify their identity andmake an association between the enrolled biometric template and theuser's identity. A biometric template represents a signature related toa biological characteristic of an individual, for example, a person'sbulk absorption characteristic for their skin tissue. Such biologicalcharacteristics may be measured at various points on the human body forexample, at a fingertip or on the wrist or ankle. Data processing logicmay then generate a biometric template from obtained biometric sensordata and compare it with the enrolled biometric template and determinewhether the templates match.

If a match is found between both of the generated templates withcorresponding enrolled template stored in the enrolled templatesdatabase 107, then the processor 105 will produce an authenticationoutput signal 109 which will enable access to the protected system 111.The multimodal authentication system 100 is referred to as “multimodal”in that both the PPG sensor 101 and the second biometric sensor 103 mustproduce a match with the enrolled templates database 107. The biometricfeatures extracted from the PPG sensor 101 that are used to generate thebiometric template is limited to the bulk absorption of the user whichis related to a DC component of the PPG sensor 101 output. In otherwords, only one of three possible components of the PPG data isutilized.

A PPG signal may be considered to consist of three components; acomponent related to arterial blood volumetric changes (which can berelated to heart activity); a component related to venous blood volumechanges (which is a slow rate signal that modulates the overall PPGsignal); and a DC component which may be related to the opticalproperties of the biological tissue. These optical properties include,for example absorptivity and reflectivity which may be mathematicallymodeled as transmission and reflection coefficients within the spectralfrequency range of interest.

A method of operation of the system shown in FIG. 1 is illustrated inthe flowchart of FIG. 2. The method of operation begins, and inoperation block 201 the processor 105 obtains first biometric data usingPPG sensor 101. In operation block 203, the processor 105 extracts bulkabsorption features from the first biometric data and generates a firstbiometric template. In operation block 205, the processor 105 obtainssecond biometric data from second biometric sensor 103. In operationblock 207 the processor 105 extracts second biometric features from thesecond biometric data and generates a second biometric template. Inoperation block 209, the processor 105 compares the generated firstbiometric template and the generated second biometric template torespective enrolled biometric templates contained in the database 107.In decision block 211, the processor 105 checks whether both of thefirst biometric template and the second biometric template form a matchwith templates contained in the database 107. If a match is found, thenin operation block 215 the processor 105 provides the authenticationoutput signal 109 and the method of operation ends as shown. If a matchwith the database 107 is not found in decision block 211, then theprocessor 105 denies access to the protected system 111 as shown inoperation block 213, and the method of operation terminates.

FIG. 3 is an example plot of photoplethysmography sensor voltage versusLED current which corresponds to bulk absorption for various users. ThePPG sensor 101 will include at least one LED and a photodiode. In someembodiments, the PPG sensor 101 will include multiple LEDs and multiplephotodiodes. The FIG. 3 example plot may be considered to correspond toa single LED and photodiode pair in a biometric PPG sensor used toobtain the measurement. Each one of the curves shown in the example plotis related to a specific user's bulk absorption characteristic. As canbe seen from the example plot, individual users have bulk absorptioncharacteristics that are different enough from each other such that adetermination of which user is which can be ascertained. This PPG bulkabsorption characteristic information is therefore useful in combinationwith other biometric data in a multimodal biometric authenticationsystem as discussed briefly above with respect to FIG. 1 and FIG. 2.

Another type of multimodal biometric system that can be implementedusing the PPG bulk absorption characteristic illustrated in the exampleplot of FIG. 3 is a system using PPG sensor measurements withindifferent spectral ranges. For example, PPG measurements may be takenwithin a red spectral range and another set of PPG measurements may betaken within, for example, a green spectral range. A red spectral range(i.e. infrared) is often used for transmission PPG system measurementsbecause red light achieves better penetration through human tissue. Incontrast, a green spectral range is often used for reflective PPGmeasurements because green does not have the ability to penetratethrough human tissue as well and therefore more of the green spectrallight applied is reflected from layers of the tissue. Therefore bytaking PPG measurements using two different spectral ranges a differentPPG characteristic template can be created for each spectral range.Therefore, in one example embodiment related to FIG. 1, the secondbiometric sensor 103 may be a second PPG sensor having a spectral rangedifferent from that of PPG sensor 101. In other embodiments, only onePPG sensor may be used where the PPG sensor has the capability of usingmultiple spectral ranges.

FIG. 4 is a flowchart showing a method of operation using either asingle multispectral PPG sensor or using a first PPG sensor having afirst spectral range and a second PPG sensor having a second spectralrange in accordance with an embodiment. The method of operation will bedescribed with respect to the multimodal biometric authentication system100 shown in FIG. 1 with the second biometric sensor 103 assumed to be asecond PPG sensor with a spectral range that differs from PPG sensor101. The method of operation begins, and in operation block 401, theprocessor 105 obtains first biometric data using the PPG sensor 101 in afirst spectral range. In operation block 403, the processor 105 extractsbulk absorption features from the first biometric data and generates afirst bulk absorption biometric template. In operation block 405, theprocessor 105 obtains second biometric data using a second PPG sensor(i.e. biometric sensor 103 in this example) in a second spectral range.In operation block 407, the processor 105 extracts bulk absorptionfeatures from the second biometric data and generates a second bulkabsorption biometric template. In operation block 409, the processor 105compares the first bulk absorption biometric template and the secondbulk absorption biometric template to respective enrolled bulkabsorption biometric templates in the database 107.

In decision block 411, the processor 105 checks whether both of thefirst bulk absorption biometric template and the second bulk absorptionbiometric template match enrolled templates within the database 107. Ifa match is detected, then in operation block 415 the processor 105provides the authentication output signal 109 to allow access to theprotected system 111 and the method of operation terminates as shown. Ifa match is not detected between one or the other, or both, of thebiometric templates and an enrolled template in the database 107, thenaccess is denied as shown in operation block 413 and the method ofoperation also terminates. In other words, both PPG characteristicstaken at the different spectral ranges must each match respectiveenrolled biometric templates in order for the authentication outputsignal 109 to be provided.

Some additional example embodiments that employ the bulk absorptionbiometric characteristic along with a second biometric will now bedescribed. It is to be understood that in the various embodiments, thesecond biometric may be a second bulk absorption biometriccharacteristic measured within a spectral range different from the firstbulk absorption biometric characteristic as described above with respectto FIG. 4. It is also to be understood that other biometrics other thanbulk absorption biometric characteristics may be used such as, but notlimited to, fingerprint detection, voice print detection, an iris scan,body odor detection, body heat signature detection or some otherbiometric.

A first such example embodiment is illustrated in the block diagram ofFIG. 5. An example multimodal biometric authentication system includes afirst device 510 having a PPG sensor 501 and a second biometric sensor502, and a second device 520 that includes authentication dataprocessing logic 515. The first device 510 includes a transceiver 507operatively coupled to the controller 505. The controller 505 isoperatively coupled to a data collector 503 which is further operativelycoupled to the PPG sensor 501 and to the second biometric sensor 502.The data collector 503 is operative to obtain raw sensor data from thePPG sensor 501 and from the second biometric sensor 502. The controller505 is operative to communicate with the transceiver 507 to pass the rawsensor data to the second device 520 over a wireless link 530 betweenthe transceiver 507 and the transceiver 509 within the second device520.

The transceiver 509, in the second device 520, is operatively coupled toa controller 511. The controller 511 is further operatively coupled toauthentication data processing logic 515. The controller 511 isoperative to receive the raw sensor data from transceiver 509 sent to itfrom the first device 510. The controller 511 hands the raw sensor datato the authentication data processing logic 515 which proceeds toextract biometric features and perform template generation. Theauthentication data processing logic 515 is also operative to comparethe generated templates to enrolled templates stored in the database513. If a match is found, then the authentication data processing logic515 also produces the authentication output signal 517. Theauthentication output signal 517 may be returned to the first device 510over the wireless link 530, may be sent to a third device (not shown),or may enable access to a protected system included on the second device520. For example, in some embodiments, the authentication output signal517 unlocks the second device 520, the first device 510, or bothdevices.

In some embodiments, the first device 510 may be categorized as a“wearable device.” As mobile devices decrease in size due to continuingadvances in miniaturization technologies, some have become “wearabledevices” in the sense that these devices may be worn by a user as afashion accessory such as jewelry, an article of clothing, a portion ofan article of clothing, etc. A wearable device may have any suitablestructure and therefore the possible wearable devices may include aring, a wristwatch (also referred to as a “smartwatch”), a button orbrooch which may include a pin for attaching to clothing, or a patchthat may be sewn to, or into, clothing such as a shirt or blouse, etc.Other example wearable devices may include a bracelet, an anklet, a beltbuckle, etc.

The wireless transceiver 507 of the wearable device, and the wirelesstransceiver 509 of the second device, may utilize any suitable wirelesstechnology such as IEEE 802.11 (also referred to as WiFi®), Bluetooth®,Wireless USB, ZigBee, Bluetooth® Low Energy (also referred to as“Bluetooth® Smart”) or any other suitable wireless technology that mayform the wireless link 530 between the first device 510 and the seconddevice 520 to transfer information or command and control signalingthere-between. The first device 510 and the second device may go througha pairing procedure or a connection procedure depending on the wirelesstechnology employed. The second device 520, may be a mobile device suchas, but not limited to, a mobile phone (also referred to as a“smartphone”), a laptop computer, electronic book reader, personaldigital assistant (PDA), electronic game console, or similar device. Insome embodiments, one of the devices may be at a fixed location and theother device may be a mobile device or wearable. Some examples of thisare when the multimodal authentication system is used for door access,or other fixed system access such as access to a fixed control panel orfixed computer equipment. In these cases, the user may have a portabledevice (i.e. the wearable or mobile device) which communicates with afixed device in the two-step authentication process.

Therefore the wireless transceivers of both devices can each receivewireless signals from, and send wireless signals to, the other devicewireless transceiver over the wireless link 530. The data collector 503is operatively coupled to the transceiver 507 such that it may send dataover the wireless link 530 to the second device 520. The data collector503 is also operative to receive command and control signals from thesecond device 520 by way of the transceiver 507 and the wireless link530. For example, a controller 511 within the second device 520 may senda command signal to the data collector 503 to initiate data collectionfrom the PPG sensor 501 or from the second biometric sensor 502.

The authentication process may be initiated in the various embodimentsusing a variety of approaches. In one approach, the PPG sensor 501 maysend out a reflective signal periodically to determine whether a user ispresent (i.e. whether human tissue is nearby and being detected).Because the first device 510, the second device 520, or both may bepowered by batteries within the devices in some embodiments, thisapproach is performed periodically in order to conserve battery powerfor the respective device. For a fixed device where power may beprovided from a power source other than a battery, power conservationmay not be an issue and therefore the PPG sensor 501 or second biometricsensor 502 may be in a constant “on” state (or “sensing state”) suchthat data collection will begin as soon as the controller 505 determinesthat human tissue is present or that the relevant inputs are present forbiometric sensor 502. In other embodiments, a third sensor may be usedin one or both devices such as a proximity sensor that detects thepresence of a user. In that case, the proximity sensor output may beused as a trigger input to the controller 505 to initiate the datacollector 503 to collect biometric sensor data from the PPG sensor 501and the second biometric sensor 502. Another approach is a userinitiated authentication where the user invokes a user interface on oneof the devices to manually initiate the authentication process. Anyother approach to initiation of the authentication process may be usedin the various embodiments.

The flowchart of FIG. 6 provides a method of operation of the multimodalauthentication system shown in FIG. 5, in accordance with an embodiment.The method of operation begins and in operation block 601, the datacollector 503 obtains first biometric data using PPG sensor 501, andobtains second biometric data using the second biometric sensor 502. Inoperation block 603, the controller 505 uses the transceiver 507 towirelessly transmit the first biometric data and the second biometricdata to the second device 520 over the wireless link 530. The controller511 receives the biometric data and passes it to the authentication dataprocessing logic 515. In operation block 605, the authentication dataprocessing logic 515 extracts the bulk absorption features from thefirst biometric data and generates a first biometric template. Inoperation block 607, the authentication data processing logic 515extracts the second biometric features from the second biometric dataand generates a second biometric template. In operation block 609, theauthentication data processing logic 515 compares the first biometrictemplate and the second biometric template to respective correspondingenrolled biometric templates contained in the database 513. In decisionblock 611, the authentication data processing logic 515 determineswhether the first biometric template and the second biometric templateboth match the database. If yes, then in operation block 615, theauthentication data processing logic 515 provides the authenticationoutput signal 517 and the method of operation terminates. If a matchwith the database is not found in decision block 611, then in operationblock 613 access is denied and the method of operation also terminates.

Another embodiment of a multimodal biometric authentication system isillustrated in FIG. 7. A first device 710 includes a PPG sensor 701 anda second biometric sensor 702 which are similar to the sensors in theFIG. 5 system. The biometric sensors are operatively coupled to datacollection and processing logic 703. The data collection and processinglogic 703 may be considered as an integration of the data collector 503features and the authentication data processing logic 515 features intoa single logic component.

The data collection and processing logic 703 is further operativelycoupled to controller 705 which is further operatively coupled totransceiver 707. The second device 720 includes a transceiver 709operative to communicate with the first transceiver 707 by wireless link630. The transceiver 709 is operatively coupled to controller 711 andthe controller 711 is operatively coupled to database 713. Thecontroller 711 is operative to provide the authentication output signal717. The transceivers and controllers in FIG. 7 are also similar to thetransceivers and controllers in the FIG. 5 system.

The flow chart of FIG. 8 provides a method of operation of themultimodal authentication system shown in FIG. 7, in accordance with anembodiment. The method of operation begins and in operation block 801,the data collection and processing logic 703 obtains first biometricdata using the PPG sensor 701 and obtains second biometric data usingthe second biometric sensor 702. In operation block 803, the datacollection and processing logic 703 extracts bulk absorption featuresfrom the first biometric data and generates a first biometric template.In operation block 805 the data collection and processing logic 703extracts second biometric features from the second biometric data andgenerates a second biometric template. In operation block 807, thecontroller 705 communicates with the transceiver 707 and wirelesslytransmits the first biometric template and the second biometric templateto the second device 720 using the wireless link 630. In operation block809, the second device controller 711 compares the first biometrictemplate and the second biometric template to respective correspondingenrolled biometric templates in the database 713. In decision block 811,if the controller 711 finds that the first biometric template and thesecond biometric template match the database 713, then in operationblock 815, the controller 711 provides the authentication output signal717 and the method of operation terminates. If a match is not found indecision block 811, then the controller 711 denies access as shown inoperation block 813 and the method of operation also terminates.

Another embodiment of a multimodal biometric authentication system isillustrated in the block diagram of FIG. 9. The multimodal biometricauthentication system includes a first device 910 and a second device920. The first device 910 includes a transceiver 907 operatively coupledto a controller 905 which is further operatively coupled to datacollection and processing logic 903. The data collection and processinglogic 903 is operatively coupled to a biometric sensor 901 and to adatabase 913. The biometric sensor 901 may be a PPG sensor or some otherbiometric sensor. The second device 920 includes a transceiver 909 thatis operative to communicate with transceiver 907 using the wireless link930. The transceiver 909 is operatively coupled to a controller 911which is further operatively coupled to data collection and processinglogic 915. The data collection and processing logic 915 is operativelycoupled to the biometric sensor 902 and to the database 917. Thebiometric sensor 902 may also be one or the other of a PPG sensor orsome other biometric sensor. The transceivers, controllers and datacollection and processing logic components are similar to thosediscussed with respect to the FIG. 5 and FIG. 7 systems. In the FIG. 9system, the first device 910 and the second device 920 eachindependently evaluate a respective biometric corresponding to the typeof biometric sensor incorporated into the device. At least one of thetwo devices incorporates a PPG sensor as its biometric sensor. The otherdevice may have either some other type of biometric sensor, or a secondPPG sensor that operates in a different spectral range from that of itspartner device.

Operation of the multimodal biometric authentication system shown inFIG. 9 is best understood with respect to the flowchart of FIG. 10. Themethod of operation begins and in operation block 1001, the datacollection and processing logic 903 obtains first biometric data usingbiometric sensor 901 which may be a PPG sensor. The second device datacollection and processing logic 915 likewise obtains second biometricdata from biometric sensor 902 which may be another type of biometricsensor or may be a second PPG sensor with a different spectral range. Inoperation block 1003, in an embodiment in which the biometric sensor 901is a PPG sensor, the data collection and processing logic 903 extractsbulk absorption features from the first biometric data and generates afirst biometric template. In operation block 1005, the data collectionand processing logic 903 compares the first biometric template to atleast one enrolled biometric template in the database 913. In decisionblock 1007, the data collection and processing logic 903 checks whetherthe biometric template matches an enrolled template in the database 913.If a match with the database 913 found in decision block 1007, then inoperation block 1011 the controller 905 communicates with thetransceiver 907 and wirelessly transmits a first authentication outputsignal to the second device 920 using the wireless link 930.

However if a match with the database 913 is not found in decision block1017, then in operation block 1009, the first device 910 wirelesslytransmits an access denied signal to the second device using thewireless link 930, and the method of operation terminates. If inoperation block 1011, the authentication output signal is transmitted tothe second device 920 then the method of operation proceeds to operationblock 1013. In operation block 1013, the second device data collectionand processing logic 915 extracts second biometric features from thesecond biometric data and generates a second biometric template. Asshown in operation block 1015, the data collection and processing logic915 then compares the second biometric data to at least one enrolledbiometric template in the database 917. In decision block 1017, the datacollection and processing logic 915 checks whether the second biometrictemplate matches an enrolled template contained in the database 917. Ifa match is found in decision block 1017, then in decision block 1019 thedata collection and processing logic 915 checks whether or not anauthentication output signal has been received from the first device910. If the authentication output signal was received in operation block1011 and verified in decision block 1019, then the data collectionprocessing logic 915 provides an authentication output signal 919 asshown in operation block 1021 and the method of operation terminates.

If the second biometric template does not match an enrolled templatecontained in the database 917, then the data collection and processinglogic 915 issues the access denied signal as shown in operation block1023 and the method of operation terminates. Likewise, if anauthentication output signal was not received from first device indecision block 1019 the access denied command is also issued inoperation block 1023 which terminates the method of operation. In otherwords, the multimodal authentication system requires that the biometricstaken by both the first device 910 and the second device 920 matchenrolled biometric templates in their respective databases.

Another embodiment of a multimodal biometric authentication system isillustrated in FIG. 11. A device 1100 includes a PPG sensor 1109, audioequipment 1105 that includes at least one microphone and that isoperatively coupled to a voice recognition engine 1103, and datacollection and processing logic 1107. The data collection and processinglogic 1107 and the voice recognition engine 1103 are both operativelycoupled to a controller 1101. The controller 1101 is operative to issuean authentication output signal 1111 to a protected system 1119 which inresponse, grants a user access. The audio equipment 1105 may include,among other things, at least one microphone, at least one speaker,signal amplification, analog-to-digital conversion/digital audiosampling, echo cancellation, and other audio processing, etc., which maybe applied to one or more microphones and/or one or more speakers of thedevice 1100.

The protected system 1119 may be a system present on the device 1100 ormay be an external system that is physically external from the device1100 in some embodiments. The device 1100 also includes a display 1113which is operatively coupled to the controller 1101, and other userinterfaces 1115 which are also operatively coupled to the controller1101. The other user interfaces 115 may include, but are not limited to,a keypad, touch sensors, a gyroscope and accelerometer (which may beseparate or integrated in a single package), mouse, buttons, etc. Thecontroller 1101 is operatively coupled to a non-volatile non-transitorymemory 1117.

The memory 1117 includes executable code 1121 for the voice recognitionengine (VRE) 1103, data collection and processing executable code 1125for the data collection and processing logic 1107, various voice printtemplates 1123 (also referred to as “voice models”) and PPG templates1127 which are biometric templates. The voice print templates 1123 andthe PPG templates 1127 are enrolled biometric template databases thatcontain biometric templates of at least one user that have been enrolledthrough a biometric enrollment process and such enrollment processes arewell understood.

The voice print templates 1123 are used by the controller 1101 tocompare a voice print obtained from the voice recognition engine 1103and determine if a match exists. Likewise, the PPG templates 1127contained in memory 1117 are also used by the controller 1101 to comparewith PPG templates generated by the data collection and processing logic1107 to determine whether a match exists. If the controller 1101 detectsa matching voice print from the voice recognition engine 1103 andmatching a PPG template from the data collection and processing logic1107, then the controller 1101 issues the authentication output signal1111 to the protected system 1119 and thereby grants access.

Operation of the device 1100 is best understood from the flowchart shownin FIG. 12. The method of operation begins and in operation block 1201,the data collection and processing logic 1107 obtains PPG biometric datausing the PPG sensor 1109. In operation block 1203, the data collectionand processing logic 1107 extracts bulk absorption features from the PPGbiometric data and generates a biometric template. In operation block1205, the controller 1101 compares the biometric template to at leastone enrolled biometric template in the PPG templates 1127 database. Inoperation block 1207, voice recognition engine 1103 obtains voice datausing at least one microphone of the audio equipment 1105. In operationblock 1209, the voice recognition engine 1103 extracts voice recognitionfeatures from the voice data and generates a voice print. In operationblock 1211, the controller 1101 compares the voice print to at least oneenrolled voice print in the voice print templates 1123. In decisionblock 1213, the controller 1101 checks whether the biometric templateand the voice print match their respective databases. If both matchesare found in decision block 1213, then in operation block 1217 thecontroller 1101 provides the authentication output signal 1111 and themethod of operation terminates with an access grant. If one or both ofeither the voice print or the biometric template (i.e. bulk absorptionfeatures) do not match their respective databases in decision block1213, then the controller 1101 denies access as shown in operation block1215 and the method of operation also terminates.

It is to be understood that the block diagrams provided herein show atleast those components necessary to describe the features and advantagesof the various embodiments to those of ordinary skill, but that variousother components, circuitry, logic, etc. may be present in order toimplement various functions and features of the devices such as theexample wearable and other mobile devices and that those various othercomponents, circuitry, devices, etc., are understood to be present inthe various embodiments by those of ordinary skill.

It is to be understood that information and commands such as, but notlimited to, raw biometric sensor data, biometric templates and/or anauthentication output signal, sent over the wireless links describedherein may be encrypted using any of various suitable data encryptiontechniques for sending data and/or for communication over wireless linksin order to enhance security and prevent middle-man snooping (alsoreferred to as “sniffing”) attacks that attempt to intercept and obtaindata over the wireless link.

The various components shown and described in the figures correspondingto multimodal authentication systems may be implemented independently assoftware and/or firmware executing on one or more programmableprocessors, and may also include, or may be implemented independently,using ASICs, DSPs, hardwired circuitry (logic circuitry), orcombinations thereof. That is, any of the various described controllers,the data collector 503, authentication data processing logic 515, datacollection and processing logic 703, data collection and processinglogic 903, data collection and processing logic 915, voice recognitionengine 1103, and/or data collection and processing logic 1107components, may be implemented using an ASIC, DSP, executable codeexecuting on a processor, logic circuitry, or combinations thereof.

Each of the various described database components are implemented in thevarious embodiments via non-volatile, non-transitory computer readablememory. The memory 1117, which is also non-volatile, non-transitorycomputer readable memory, contains executable instructions or executablecode, such as VRE executable code 1121 and data collection andprocessing executable code 1125, for execution by at least oneprocessor, that when executed, cause the at least one processor tooperate in accordance with the functionality and methods of operationherein described. The computer readable memory may be any suitablenon-volatile, non-transitory, memory such as, but not limited to,programmable chips such as EEPROMS, flash ROM (thumb drives), compactdiscs (CDs) digital video disks (DVDs), etc., that may be used to loadexecutable instructions or program code to other processing devices suchas mobile devices, including wearable devices, or other devices such asthose that may benefit from the features of the herein describedembodiments.

Various applications exist for the multimodal biometric systemsdisclosed herein and these various applications are contemplated by thepresent disclosure. One such application is to add a PPG sensor having agreen spectral range for obtaining a reflective PPG measurement to amedical device having an infrared PPG sensor. The green spectral rangePPG measurement may be used to identify or authenticate the patient andto correlate their infrared PPG measurements to their patient records.The patient would have been previously enrolled in the PPG biometricdatabase for the hospital. Another application is a device unlockingfeature that requires a PPG biometric on, for example, a smartwatchhaving a PPG sensor incorporated in the wristband or the case housing,and a second biometric such as a voice print from a second device suchas a smartphone. The smartwatch may send an authentication output signalwirelessly to the smartphone to unlock the smartphone if the user's PPGbulk absorption characteristic taken at the smartwatch match and a voiceprint taken by the smartphone also matches. Other ways of combining twoor more biometric features to enhance security and authenticationreliability will occur to those of ordinary skill in light of thepresent disclosure.

While various embodiments have been illustrated and described, it is tobe understood that the invention is not so limited. Numerousmodifications, changes, variations, substitutions and equivalents willoccur to those skilled in the art without departing from the scope ofthe present invention as defined by the appended claims.

What is claimed is:
 1. A wearable device comprising: aphotoplethysmography (PPG) sensor; a second biometric sensor; atransceiver; a controller, operatively coupled to the PPG sensor, to thesecond biometric sensor, and to the transceiver, the controlleroperative to: obtain biometric data collected using the PPG sensor andusing the second biometric sensor; and send the biometric data obtainedusing the PPG sensor and using the second biometric sensor to a seconddevice over a wireless link using the transceiver.
 2. A multimodalbiometric authentication system comprising the wearable device of claim1, and further comprising: a mobile device wherein the mobile device isthe second device, the mobile device comprising: a transceiver; acontroller, operatively coupled to the transceiver; non-volatile,non-transitory memory, containing at least two enrolled biometrictemplates, a first enrolled biometric template related to a bulkabsorption biometric and a second enrolled biometric template related toa biometric measurable by the second biometric sensor in the wearabledevice; data processing logic, operatively coupled to the controller,the data processing logic operative to: extract bulk absorption featuresfrom biometric data obtained using the PPG sensor in the wearable deviceand generate a first biometric template; extract biometric features frombiometric data obtained using the second biometric sensor in thewearable device and generate a second biometric template; and provide anauthentication output signal in response to the first biometric templatematching the first enrolled biometric template and the second biometrictemplate matching the second enrolled biometric template.
 3. Thewearable device of claim 1, wherein the second biometric sensor is abiometric sensor selected from the group consisting of: a second PPGsensor operative in a second spectral range different from the other PPGsensor, a microphone, a fingerprint sensor, a palm reader, and an irisscanner.
 4. A method comprising: obtaining biometric data collectedusing a photoplethysmography (PPG) sensor and using a second biometricsensor, both the PPG sensor and the second biometric sensor in awearable device; and sending the biometric data obtained using the PPGsensor and using the second biometric sensor to a second device over awireless link using a transceiver in the wearable device.
 5. The methodof claim 4, further comprising: receiving the biometric data by a mobiledevice; extracting bulk absorption features from biometric data obtainedusing the PPG sensor in the wearable device and generating a firstbiometric template; extracting biometric features from biometric dataobtained using the second biometric sensor in the wearable device andgenerating a second biometric template; and providing an authenticationoutput signal in response to the first biometric template matching afirst enrolled biometric template and the second biometric templatematching a second enrolled biometric template.
 6. A method comprising:sending biometric data obtained using a photoplethysmography (PPG)sensor and using a second biometric sensor from a first device to asecond device over a wireless link; extracting bulk absorption featuresin the second device, from the biometric data obtained using the PPGsensor in the first device and generating a first biometric template inthe second device; extracting biometric features in the second device,from the biometric data obtained using a second biometric sensor in thefirst device and generating a second biometric template in the seconddevice; and providing an authentication output signal in response to thefirst biometric template matching a first enrolled biometric templatestored in the second device and the second biometric template matching asecond enrolled biometric template stored in the second device.
 7. Awearable device comprising: a photoplethysmography (PPG) sensor; asecond biometric sensor; a transceiver; data processing logic,operatively coupled to the PPG sensor and to the second biometricsensor, the data processing logic operative to: extract bulk absorptionfeatures from biometric data obtained using the PPG sensor and generatea first biometric template; and extract biometric features frombiometric data obtained using the second biometric sensor and generate asecond biometric template; a controller, operatively coupled to the dataprocessing logic, and to the transceiver, the controller operative to:send the first biometric template and the second biometric template to asecond device over a wireless link using the transceiver.
 8. Amultimodal biometric authentication system comprising the wearabledevice of claim 7, and further comprising: a mobile device wherein themobile device is the second device, the mobile device comprising: atransceiver; non-volatile, non-transitory memory, containing at leasttwo enrolled biometric templates, a first enrolled biometric templaterelated to a bulk absorption biometric and a second enrolled biometrictemplate related to a biometric measurable by the second biometricsensor in the wearable device; and a controller, operatively coupled tothe transceiver and to the memory, the controller operative to providean authentication output signal in response to the first biometrictemplate matching the first enrolled biometric template and the secondbiometric template matching the second enrolled biometric template. 9.The multimodal biometric authentication system of claim 8, wherein thecontroller is further operative to: send the authentication outputsignal over a wireless link using the transceiver in the mobile device.10. A method comprising: extracting bulk absorption features frombiometric data obtained using a photoplethysmography (PPG) sensor in afirst device and generating a first biometric template in the firstdevice; extracting biometric features from biometric data obtained usinga second biometric sensor in the first device and generating a secondbiometric template in the first device; and sending the first biometrictemplate and the second biometric template from the first device to asecond device over a wireless link.
 11. The method of claim 10, furthercomprising: providing an authentication output signal in response to thefirst biometric template matching a first enrolled biometric templatestored in the second device and the second biometric template matching asecond enrolled biometric template stored in the second device.
 12. Themethod of claim 10, wherein extracting biometric features from biometricdata obtained using a second biometric sensor and generating a secondbiometric template, comprises: extracting a voice print as the biometricfeatures from biometric data obtained using a microphone as the secondbiometric sensor and generating a voice print as the second biometrictemplate.
 13. A multimodal biometric authentication system comprising: afirst device and a second device, the first device and the second deviceeach comprising: a biometric sensor; a transceiver; non-volatile,non-transitory memory, containing at least one enrolled biometrictemplate, with at least one of the first or the second devices having afirst enrolled biometric template related to a bulk absorption biometricand the other of the first or second devices having a second enrolledbiometric template; data processing logic, operatively coupled to thebiometric sensor, the data processing logic operative to: extractbiometric features from biometric data obtained using the biometricsensor in the respective first device and second device, and generate arespective first biometric template in the first device and a secondbiometric template in the second device, wherein the biometric featuresextracted by at least one of the first device or the second device arebulk absorption features; a controller, operatively coupled to the dataprocessing logic, and to the transceiver, the controller operative to:provide an authentication output signal in response to the respectivebiometric template matching the at least one respective enrolledbiometric template; and send the authentication output signal to theother device over a wireless link using the transceiver.
 14. Themultimodal biometric authentication system of claim 13, wherein thebiometric sensor of the first device is a photoplethysmography (PPG)sensor; and the biometric sensor of the second device is a biometricsensor selected from the group consisting of: a second PPG sensoroperative in a second spectral range different from the first device PPGsensor, a microphone, a fingerprint sensor, a palm reader, and an irisscanner.
 15. The multimodal biometric authentication system of claim 13,wherein the biometric sensor of the first device is aphotoplethysmography PPG sensor operative in a first spectral range andthe biometric sensor of the second device is a PPG sensor operative in asecond spectral range.
 16. A method comprising: extracting bulkabsorption features from biometric data obtained using aphotoplethysmography (PPG) sensor in a first device and generating afirst biometric template in the first device; sending a firstauthentication output signal, from the first device to a second deviceover a wireless link, in response to the first biometric templatematching a first enrolled biometric template stored in the first device;extracting biometric features from biometric data obtained using asecond biometric sensor in a second device and generating a secondbiometric template in the second device; and providing a secondauthentication output signal in response to receiving the firstauthentication output signal from the first device, and the secondbiometric template matching a second enrolled biometric template storedin the second device.
 17. The method of claim 16, wherein extractingbiometric features from biometric data obtained using a second biometricsensor in a second device and generating a second biometric template inthe second device, comprises: extracting biometric features selectedfrom the group consisting of: a voice print, a fingerprint, a palmprint, and an iris scan.
 18. The method of claim 16, wherein extractingbiometric features from biometric data obtained using a second biometricsensor in a second device and generating a second biometric template inthe second device, comprises: extracting bulk absorption features frombiometric data obtained using the second biometric sensor, where thesecond biometric sensor is a second PPG sensor in the second device andoperates in a different spectral range from the first PPG sensor in thefirst device; and generating the second biometric template in the seconddevice using the extracted bulk absorption features from the biometricdata obtained at the different spectral range.